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Claims 

1. An interactive spatialized audiovisual system for linking a plurality of remote 
user terminals, the system comprising: 

- a networked computer; 

- an associated user database including user status information; 

- input means for receiving at the computer a plurality of audio streams and associated 
locating data from the remote user terminals for virtually locating the users relative to one 
another within a virtual user environment; 

- selection means for enabling selection of at least a first group of the audio streams in a 
first selection process based on status information in the user database; 

- output means for outputting thfe selected group of audio streams and associated locating 
data for spatialization of the selected group of audio streams relative to a first listener-based 
audio reference frame which is substantially coherent with yisual representations of the audio 
sources defined by the locating data at a first user terminal. 

2. A system according to claim 1 which includes first spatiaUzation means for 
spatializing the selected group of audio streams, 

3. A sj^tem according to either one of the preceding claims which includes merging . 
means for merging at least some of the audio streams into a merged audio stream for transmittal 
to the user terminal, and second spatializing means for spatializing the merged stream so as to 
provide for a spatialized background audio effect in the audio reference frame at the user 
terminal. 

4. A system according to any one of the preceding claims wherein the selection 
means are arranged to select different groups of audio streairus according to different selection 
processes based on the user status information in the user database, for transmission to the 
corresponding user tominals. 
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5, A system according to any one of the preceding claims wherein the user status 
information is chosen from a group including user location data which serves to locale the user 
in the virtual environment, user orientation data which serves to orientate the user both with 
respect to the other users and to the virtual environment, user listening status infomiation and 
user talking status information. 

6. A systOTi according to claim 5 wherein the user listening status information is 
arranged to allow the user to listen to other selected users or groups in the environment. 

7- A system according to either one of the preceding claims 5 or 6 in which the 
selection means includes a first selector for selecting M closest audio sources from N audio 
soiu'ces. 

8. A syst^ according to any one of the preceding claims 5 to 7 in which the 
selection means includes a second selector for enabling the selection of M loudest sources based 
on at least one of the following, namely the amplitude of the source signal and the distance of 
the source from the listener. 

9. A system according to any one of the preceding claims 5 to 8 in which the 
selection means includes a third selector for enabling a user-driven selection process determined 
by the subjiect user or other users. 

10. A system according to any one of the preceding claims 5 to 9 in which the 
selection means includes a fourth selector for enabling a moderator-driven selection process in 
which a "moderator" ui the environment is able to control the talking and listening status of the 
other users. 

11. A system according to any one of the preceding claims 5 to 10 in which the 
selection means includes a fifth selector for enabling a selection based on the geography or 
topology of the virtual environment, in which features of the environment are arranged 
realistically to affect the listening capability of users in the environment,* so as to provide a 
coherent visual and sonic landscapes. 

12. A system according to claim 11 wherein the features include barriers defining 
one or more chat rooms, with at least some of the audio streams in a particular room being 
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summed or merged and spatialized to achieve a background reverberation effect characteristic 
of that particular room. 

13. A system according to either claim 11 or claim 12 which includes means foi: 
merging and spatialising the audio streams in adjoining rooms or areas to create "threshold** 
effects at entrance/exit points representations of tiie combined noise emanating from said 
adjoining room or area. 

14. A system according to any one of the preceding claims II to 13 which includes 
means for generating "dry" room signals using summed non-reverberated audio sources. 

15. A syst^ according to any one of the preceding claims 10 to 13 which includes 
means for generating **wet" room signals using summed reverberated audio sources* 

16. A system according to claim 2 wherein the first spatialization means is arranged 
to process selected groups of audio streams at the networked computer to derive spatialized 
audio streams for onward transmission to at least the first selected user terminal. 

17. A system according to claim 2 wherein the iSrst spatialization means are provided 
at each of the user terminals for processing of selected groups of audio streams from the 
networked computer. 

18. A system according to any one of claims 2, 16 and 17 wherein the first 
spatialization means includes a binaural processor. 

19. • A system according to claim 3 wherein the second spatializing means is arranged 
to process the merged group of audio streams at the networked computer to derive a spatialized 
merged audio stream for onward transmission to at least the first selected liser terminal. 

20. A system according to claim 3 wherein the second spatialization means are 
provided at each of the user terminals for spatializing merged groups of audio streams at each 
user terminal. 

21. A system according to any one of claims 3, 19 and 20 wherein the second 
spatialization means includes a binaural reverberation processor. 

22. A method of providing an interactive spatialized audiovisual facility comprising: 
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- receiving from a plurality of user-based audio sources a plurality of corresponding 
audio streams and associated locating data capable of virtually locating the audio sources 
relative to one another within a virtual environment; 

- determining user status data; 

- selecting at least some of the audio streams based on the user status data; 

- transmitting the locating data and selected audio streams to a first listener destination 
for enabling the display of visual r^resentations of the virtual locations of at least some of the 
audio sources within the virtual environment, and 

- spatializing the selected audio streams relative to a first listener-based audio reference 
frame which is substantially coherent with the visual representations of the audio sources either 
before or after the audio streams are transmitted to the first listener destination. 

23. A method according to claim 22 which includes: 

- enabling the user status data to be altered, 

- reading the altered user status data, and 

- selecting at least one of the audio streams based on the altered user status data^ 

wherein at least one of the audio streams selected using the altered user status data is different 
to the prior selected streams. 

24. A method according to either one of claims 22 or 23 which includes the steps of: 

- merging at least some of the audio streams into at least one merged audio stream, 

- transmitting the merged audio stream to the first listener destination, and 

- spatializing the merged audio stream either before or after transmitting it so as to 
provide a background audio effect within the virtual environment. 

25. A method according to claim 24 in which the merged audio stream includes 
audio streams which have not been individually selected. 

26. A method of providing an interactive spatialized audiovisual facility comprising: 
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- receiving from a plurality of user-based audio sowces a plurality of corresponding 
audio streams and associated locating data capable of virtually locating the audio sources 
relative to one another within a virtual environment; 

- determining user status data; 

- selecting at least some of the audio streams based on the user status data in a first 
selection process; 

- transmitting the selected audio streams and associated locating data to a first listener 
destination for enabling the display of visual representations of the virtual locations of at least 
some of the selected audio sources within the virtual environment; 

- spatializing the selected audio streams relative to a first listener-based audio refer^ce 
jSrame which is substantially coherent v^th the visual representations of the audio sources either 
before or after the transmitting said streams; 

- selecting at least some of die audio streams in a second selection process; and 

- transmitting the selected audio streams and associated locating information to a second 
listener destination for enabling the display of visual representations of the locations of at least 
the selected audio sources, and spatializing at the second listener destination the selected audio 
streams in an audio reference frame which is substantially coherent witti the visual 
representations of the audio sowces, either before or after transmitting said streams. 

27. A method according to claim 26 in which multiple selection processes are used 
to select the audio streams according to at least one predetermined algorithm, die selected audio 
streams and associated locating information are transmitted to multiple listener destinations, and 
visible representations of the locations of at least the selected audio streams are displayed at the 
multiple listener destinations, with each of the selected audio streams being spatialized at the 
multiple listener destinations or before they are transmitted thereto in audio reference frames 
which are substantially coherent with the visible representations of the audio sources. 

28. A system for providing for spatialized conversation oveir a network environment, 
the system comprising: 

- a plurality of user terminals; 
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- a computer network capable of streaming audio streams to the user terminals, each of 
the audio streams including associated spatialization information; 

- a rendering system for rendering the audio streams to predetermined virtual locations 
around a user using the associated spatialization information; and 

- a user interface for virtually spatially locating a user amongst the audio streams; 

wherein the rendering system spatializes the audio streams so as to maintain a 
substantially spatially coherent audio reference frame around the user, the user interface 
includes a visual indicator of the spatial position of each of the audio streams around a listener 
and the rendering system maintains a .substantially spatially coherent audio reference frame 
relative to the visual indicator. 

29. A system according to claim 28 in which each stream includes user ownership 
information and the system includes an audio stream access interface for granting access to the 
audio streams. 

30. A system according to either one of preceding claims 28 or 29 which the 
rendering system includes attenuating means for attenuating audio source located virtually 
remotely from a current user and merging means foi* merging audio sources located virtually 
remotely from a current user, 

31. A system according to claim 30 in which the rendering system is located adjacent 
a user and the audio sources are streamed over a computer network. 

32. A system according to any one of the preceding claims 1 to 21 wherein the 
virtual user enviroimient is a chat room environment. 

33. A computer-readable medium having stored thereon executable instructions for 
causing a computer to provide an interactive spatialized audiovisual facility, the instructions 
being arranged to: 

- receive from a plurality of user-based audio sources a plurality of corresponding audio 
streams and associated locating data capable of virtually locating the audio sources relative to 
one another within a virtual environment; 

- determine user status data; 
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- select at least some of the audio streams based on the user status data; 

- transmit the locating data and selected audio streains and associated to a first listener 
destination for enabling the display of visual representations of the virtual locations of at least 
some of the audio sources within the virtual environment, and 

- spatialize the selected audio streams relative to a first listener-based audio reference 
frame which is substantiaUy coherent with the visual representations of the audio sources. 

34. A computer-readable medium having stored thereon executable instmctions for 
causing a computer to provide an interactive spatialized audiovisual facility, the instructions 
being arranged to: 

- receive fix>m a plurality of user-based audio sources a plurality of corresponding audio 
streams and associated locating data capable of virtually locating the audio sources relative to 
one another within a virtual environment; 

- determine user status data; 

- select at least some of the audio streams based on the user status data in a first selection 
process; 

- transmit the selected audio streams and associated locating data to a first listener 
destination for enabling the display of visual representations of the virtual locations of at least 
some of the selected audio sources within the virtual environment; 

- spatialize the selected audio streams relative to a first listener-based audio reference 
frame which is substantially coherent with the visual representations of the audio sources; 

- select at least some of the audio streams in a second selection process; and 

- trmsmit the selected audio streams and associated locating information to a second 
listener destination for enabling the display of visual representations of the locations of at least 
the selected audio sources, and spatializing at the second listener destination the selected audio 
streams in an audio reference frame which is substantially coherent with the visual 
representations of the audio sources. 



wo 03/058473 



26 



PCT/AU03/00019 



35. A method of operating an interactive spatialized audio facility including a 
networked computer and a plurality of user terminals linked to the networked computer, the 
method comprising: 

- transmitting from a user terminal to the networked computer an audio stream generated 
by the user and associated locating data capable of virtually locating the audio stream generated 
by the user within a virtual environment for selective combination with corresponding audio 
streams, associated locating data and user status data at the networked computer; 

- receiving at the user terminal a plurality of audio streams selected on the basis of the 
user status data and associated locating data for virtually locating the users relative to one 
another within a virtual user enviromnent; 

- generating at the user terminal visual representations of the locating data, and 

- spatializing the selected group of audio streams relative to a user based audio reference 
frame which is substantially coherent with the visual representations of the audio sources of the 
users as defined by the locating data for playback to the user. 

36. A method according to claim 35 which includes receiving at the user terminal a 
merged audio stream which is spatialized before or aiter receipt thereof to provide a spatialized 
background audio effect in the audio reference frame at the user terminal for playback to the 
user. 

37. A computer-readable medium having stored thereon executable instructions for 
causing a computer to provide or operate an interactive spatialized audiovisual facility, the 
instructions including program segments arranged to implement a method according to any one 
of claims 22 to 2^35 and 36. 



